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Figure 1: Photograph compared to a face rendered using our skin reflectance model. The rendered image was composited on top of the 
photograph. Right: Changing the albedo and BRDF using statistics of measured model parameters from a sample population. 



Abstract 



1 Introduction 



We have measured 3D face geometry, skin reflectance, and subsur- 
face scattering using custom-built devices for 149 subjects of vary- 
ing age, gender, and race. We developed a novel skin reflectance 
model whose parameters can be estimated from measurements. The 
model decomposes the large amount of measured skin data into a 
spatially-varying analytic BRDF, a diffuse albedo map, and diffuse 
subsurface scattering. Our model is intuitive, physically plausible, 
and - since we do not use the original measured data - easy to edit 
as well. High-quality renderings come close to reproducing real 
photographs. The analysis of the model parameters for our sam- 
ple population reveals variations according to subject age, gender, 
skin type, and external factors (e.g., sweat, cold, or makeup). Using 
our statistics, a user can edit the overall appearance of a face (e.g., 
changing skin type and age) or change small-scale features using 
texture synthesis (e.g., adding moles and freckles). We are making 
the collected statistics publicly available to the research community 
for applications in face synthesis and analysis. 

Keywords: Face Modeling, Reflection Models, Data-Driven Mod- 
els 
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One of the most difficult computer graphics challenges is creating 
digital faces that are indistinguishable from real ones. The process 
of capturing an actor's performance and likeness has accurately 
been named "Digital Face Cloning." Digital face cloning has many 
applications in movies, games, medicine, cosmetics, computer vi- 
sion, biometrics, and virtual reality. Despite recent progress, how- 
ever, it remains a challenging problem because humans are incred- 
ibly sensitive to the subtleties of real faces. 

In this paper, we focus on modeling skin reflectance of human 
faces, an important aspect of face appearance. Our primary goal is 
to accurately capture and reproduce the face reflectance of a person. 
We use custom-built devices to measure 3D face geometry, light re- 
flection, and subsurface scattering. Our reflectance model consists 
of a spatially- varying analytic reflectance model (Torrance-Sparrow 
or Bl inn- Phong), a diffuse albedo map, and diffuse scattering coef- 
ficients. We concisely describe the mathematics of the model and 
show how its components can be robustly estimated from measured 
data. The model is compact, intuitive, and easy to use. Images gen- 
erated from it come very close to real photographs (see Figure 1). 

A second goal is to capture and describe the variation of face 
reflectance for a sample population. We measured the parameters 
of our reflectance model for 149 people of different race, gender, 
and age. The result is a unique database of spatially-varying para- 
meters for the Torrance-Sparrow and B I inn-Phong reflectance mod- 
els, histograms of skin albedo, and diffuse scattering parameters for 
faces. From this data we computed the distributions of the parame- 
ters (most of them are Gaussian) and analyzed them using statis- 
tical methods. We publish the key findings of our analysis in this 
paper, and make the complete statistical data available to the re- 
search community. To our knowledge, this is the first study of face 
reflectance of this kind. 

The results of our analysis have applications in realistic face syn- 
thesis, face reconstruction from photographs, inverse rendering, and 
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computer vision. We also demonstrate how an artist can edit faces 
using the parameter distributions. For example, we can change 
the skin type of a model by transferring statistics of albedo and 
reflectance parameters (see Figure 1). We can also use texture syn- 
thesis/analysis to transfer small-scale skin features, such as moles 
and freckles. The combination of these approaches allows an artist 
to convincingly transfer face appearance between individuals. 

2 Previous Work 

Properties of human skin have been measured and studied exten- 
sively in the computer graphics, biomedical, and computer vi- 
sion communities. An excellent survey on the physiological and 
anatomical properties of skin and the state of the art in skin appear- 
ance modeling has been published by Igarashi et al. [2005]. In this 
section we provide an overview of the relevant work in the area of 
face appearance modeling in computer graphics. 

Analytic Face Appearance Models: The simplest reflectance 
model for skin is the Bidirectional Reflectance Distribution Func- 
tion (BRDF) [Nicodemus et al. 1977]. Dana et al. [1999] 
measured the BRDF of skin and made the data available in 
the Columbia-Utrecht Reflectance and Texture (CuReT) database 
(http://www.cs.columbia.edu/CAVE/curet/). Because the data was 
not collected in vivo it does not accurately reflect the properties 
of living skin. Marschner et al. [1999; 2000b] measured spatially 
uniform BRDFs on the foreheads of four subjects of different gen- 
der, race, and age. They observed that the BRDF of skin exhibits 
strong forward scattering in off-specular direction at grazing an- 
gles. In later work they augmented the uniform specular BRDF 
with a spatially-varying albedo texture [Marschner et al. 2000a], 
Their BRDF model was used in the human face project at Walt 
Disney Studios [Williams 2005]. That project, which had a key in- 
fluence in the historical development of human face models (e.g., 
it was influential on the process developed for the Matrix sequels), 
was completed in 2001. Other researchers have estimated para- 
meters of uniform analytic BRDF models from photographs with 
static [Blanz and Vetter 1999; Paris et al. 2003] or varying illumi- 
nation [Georghiades 2003; Fuchs et al. 2005b]. Fuchs et al.[2005b] 
cluster different parts of the face into regions. They estimate a 
spatially-varying diffuse albedo but assume specularity to be con- 
stant within each region. The fundamental limitation of all these 
models is that they ignore subsurface scattering that is largely re- 
sponsible for the soft appearance of facial skin. 

Hanrahan and Krueger [1993] were the first to model subsurface 
scattering in skin using a diffuse constant and a lighting-dependent 
single scattering term. They described scattering and absorption 
parameters of skin using the Henyey-Greenstein function and com- 
pute scattering with Monte Carlo simulations. A similar approach 
was used by Ng and Li [2001]. To avoid expensive Monte Carlo 
simulations, Stam [2001] developed an analytic approximation to 
multiple subsurface scattering for skin with a rough surface. All 
of these models describe skin reflectance using a BRDF and ignore 
light scattering between different incoming and outgoing locations 
on the surface. 

To describe the full effect of light scattering between two points 
on the surface one can use the Bidirectional Surface-Scattering Dis- 
tribution Function (BSSRDF). Krishnaswamy et al. [Krishnaswamy 
and Baranoski 2004] introduced a BSSRDF model for skin with 
spectral and biophysically-based parameters. The BSSRDF is 
eight-dimensional, making it very difficult to measure and repre- 
sent. Jensen et al. [2001] were the first to propose a practical low- 
dimensional analytic approximation for the BSSRDF of homoge- 
neous materials using diffusion theory. To estimate the parameters 
of their model they measure scattering of light in skin on a forearm 



in vivo using a white focused beam of light and a camera. We use 
their approximation for subsurface scattering and developed a mea- 
surement device that can safely be used in faces (see Section 5.4). 

Jensen and Buhler [Jensen and Buhler 2002] develop fast ren- 
dering methods for this approximation, and Hery implemented 
the technique for use in feature films [Hery 2005). Both address 
the derivation of spatially varying subsurface scattering parameters 
from an albedo map. As discussed in Section 5.4, we do not use 
spatially varying subsurface scattering coefficients and instead use 
a constant scattering term, modulated by a modulation texture. Re- 
cently, Donner and Jensen [2005] have introduced a multi-layer dif- 
fusion approximation for rendering skin. Unfortunately, measuring 
the parameters of each layer of multi-layered translucent materials 
is difficult. 

Image-Based Face Models: Image-based models have provided 
highly realistic representations for human faces because they im- 
plicitly capture effects such as self-shadowing, inter-reflections, 
and subsurface scattering. Pighin et al. [1998] use view-dependent 
texture mapping [Debevec et al. 1996] to reproduce faces under 
static illumination conditions. More recent efforts allow varia- 
tions in lighting for static faces [Georghiades et al. 1999; De- 
bevec et al. 2000], expressions [Hawkins et al. 2004], and real- 
time performances [Wenger et al. 2005]. Debevec et al. [2000] 
present a process for creating realistic, relightable 3D face mod- 
els by mapping image-based reflectance characteristics onto 3D- 
scanned geometry. In order to change the viewpoint, they use color- 
space analysis to separate the image data into specular and diffuse 
components that can be extrapolated to new viewpoints. While their 
method does consider the aggregate behavior of subsurface scatter- 
ing, they do not model a specific diffusion parameter. Thus, unlike 
our approach, their method cannot produce correct subsurface scat- 
tering effects for closeup light sources or high-frequency spatially- 
varying illumination. Borshukov and Lewis [2003] combine an 
image-based model, an analytic surface BRDF, and an image-space 
approximation for subsurface scattering to create highly realistic 
face models for the movie industry. Sander et al. [2004] developed 
a variant of this method for real-time face rendering on modern 
graphics hardware. 

Image-based models come at the price of extensive data storage. 
They are difficult to edit, and the measurements are not related to 
any physical model of light-skin interaction. It is very difficult to 
simulate local light sources and to reproduce spatially-varying illu- 
mination [Jones et al. 2005], especially if shadows from a distant 
light disagree with shadows from a local light. An example of this 
would be a light source close to the nose - even though the skin 
is illuminated, an image-based method would predict zero reflec- 
tion since a distant light from that direction would be blocked by 
the nose. In contrast to image-based methods, we measure a full 
BRDF at each surface point using a face-scanning dome similar to 
the LightStage by Debevec et al. [2002], but with more cameras 
(16). The resulting parametric face reflectance model has lower 
storage requirements than a full surface reflectance field. In addi- 
tion it is physically plausible, easy to render, and intuitive to edit. 

Skin Color and Texture Models: Biophysical studies show that 
skin appearance is largely dependent on wavelength [Igarashi et al. 
2005]. Angelopoulou et al. [2001] measured skin BRDF as a func- 
tion of wavelength for a sample population of 22 people and noted 
that five Gaussian basis functions reproduce the data well. The 
study was limited to 0° incident and 4° reflection angle. 

Tsumura et al. [2003] used Independent Component Analysis 
(ICA) in color space to decompose images of faces into melanin 
and hemoglobin layers. To change skin color and transfer skin 
texture they adapted the texture synthesis method of Heeger and 
Bergen [1995]. We use a similar approach to transfer texture (see 
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Section 8). Cula et al. [2005; 2004] collected the Bidirectional Tex- 
ture Function (BTF) for different body regions of a sample popula- 
tion (http://www.caip.rutgers.edu/rutgers_texture/cvg/). They ana- 
lyzed and classified skin texture using histograms of image-pyramid 
features [Cula and Dana 2002]. We use a similar analysis technique 
for albedo maps (see Section 7). 

3 Skin Reflectance Model 

The BSSRDF is a function 5(^,Ct) ( -,jc 0 ,ft) 0 ), where (D t is the direc- 
tion of the incident illumination at point jc,, and (0 o is the observa- 
tion direction of radiance emitted at point x 0 . The outgoing radiance 
L(x 0} (0 o ) at a surface point x Q in direction (0 o can be computed by 
integrating the contribution of incoming radiance L(jc/,6),) for all 
incident directions Q. over the surface A: 

L(x Oi (0 0 ) = ( f Sixi^i^cOo^ixi^w^N ^dcOidAixi). (1) 

J A JO. 

We will use the notation dx t for (N (Oi)dA{xi). 

We separate this integral into a specular BRDF term and a diffuse 
subsurface scattering term: 

L(x 0 , 6> 0 ) = L specular (x 0 , (0 o ) + L diffuse (x 0 , co 0 ), (2) 

with: 

^specular ( x o ? ®o ) 

Ldiffuse(Xo,«>o)= I I SdiXi^Xo^Lixi^dCOidXi. (4) 
J A JO, 

Sd(*i, ®i,Xo> ®o) is called the diffuse BSSRDF and f s (x 0 , <0 o , a*) is 
a surface BRDF. As observed in [Jensen and Buhler 2002], multiple 
scattering dominates in the case of skin and can be modeled as a 
purely diffuse subsurface scattering term and a specular term that 
ignores single scattering (see Figure 2). 



SpecularReflection (BRDF) f s 




Modulation Texture M 



; Homogeneous Subsurface 
Scattering R 

Figure 2: Spatially-varying skin reflectance can be explained by a 
specular BRDF f s at the air-oil interface, and a diffuse reflectance 
component due to subsurface scattering. We model the diffuse re- 
flectance with a spatially-varying modulation texture M and a ho- 
mogeneous subsurface scattering component R d . 

The diffuse BSSRDF S d is very difficult to acquire directly and 
is often decomposed into a product of lower-dimensional functions. 
A commonly used decomposition for S d is: 



S d (xi,(o h x 0) (o 0 ) « -fi(x h a)i)R d (x h x 0 )fo(x 0 ,a) 0 ). 

71 



(5) 



Here R d (xi,x 0 ) describes the diffuse subsurface reflectance of light 
entering at point jc,- and exiting at point x 0 . The diffuse surface 
reflectance/transmittance functions //(jc t ,fi),) and f o (x o ,(0 o ) further 
attenuate the light at the entrance and exit points based on the in- 
coming and outgoing light directions, respectively. 



The complete formula for our skin model is: 

L{x 0% (0 0 )= [ f s (x o ,Q) o ,(Qi)L(xo,(0i)(N a)i)dQ>i (6) 
Jo 

+ / / F ti R d (\\x o -Xi\\)M(x 0 )F t0 L(x ii G)i)d(0 i dx i . 
J A JO, 

Like Hanrahan and Krueger [1993] and Jensen et al. [2001], we 
assume that skin is mostly smooth and use transmissive Fresnel 
functions F ti = F t {r},(Qi) and F t0 = F t (r),(0 o ). We use rj « 1.38 as 
the relative index of refraction between skin and air for the Fresnel 
terms [Tuchin 2000]. 

We model the diffuse reflectance using a homogeneous subsur- 
face reflectance term /?</(||jc 0 — and a spatially- varying modula- 
tion texture M(x 0 ) [Fuchs et al. 2005a]. Intuitively, this modulation 
texture represents a spatially-varying absorptive film of zero thick- 
ness on top of homogeneously scattering material (see Figure 2). To 
model homogeneous subsurface scattering we use the dipole diffu- 
sion approximation by Jensen et al. [2001]. It simplifies R d (xj,x 0 ) 
to a rotationally-symmetric function R d (\\x 0 -JC;||), or R d (r) with 
radius r. 

Our formulation in Equation (6) was motivated by the skin model 
shown in Figure 2. Goesele et al. [2004] assume //(*/, (0 ( ) and 
f o (x o ,(0 o ) to be constant. To model inhomogeneous translucent 
materials, they exhaustively measure R d {xi,x 0 ) and tabulate the re- 
sults. In follow-up work [Fuchs et al. 2005a] they fit a sum of Gaus- 
sians to the tabulated values of R d (x^x Q ) to model anisotropic scat- 
tering. However, acquiring R d (xi,x a ) directly requires a substantial 
experimental setup (controlled laser illumination from different an- 
gles) that is not applicable for human faces. 

Donner and Jensen [2005] use spatially-varying diffuse trans- 
mission functions p dt (*,• ,6),) = 1 - f a f s (*,• , (tf/ , <D 0 ) ( (0 o • n )d(0 0 
(similarly for p dt {x 0 ,(0 0 )). They integrate a BRDF at the surface 
over the hemisphere of directions and tabulate the results to obtain 
the diffuse transmission functions. To capture materials with homo- 
geneous subsurface scattering and complex surface mesostructure 
(so-called quasi-homogeneous materials), Tong et al. [2005] mea- 
sure and tabulate a spatially-varying exit function f 0 {x 0 , <o 0 ) and a 
uniform entrance function /i(cty)- We now give an overview of how 
we measure and estimate the parameters of our model. 

4 Parameter Estimation Overview 

A block diagram of our processing pipeline is shown in Figure 3. 
Wc first capture the 3D geometry of the face using a commercial 3D 

/ 3D Surface / / Reflectance / / Subsurfaca / 
/ Scan / / Measurement* / / Scant / 



f Normal Map and \ f \ 
Diffuse Reflectance CHpole Model Fit ) 

V Estimation J V J 



Specular 
Reflectance 



^ BRDF Fit ^ 



sJgma, / atgma; 



SpatiE*y-Varyino 
Specular BRDF 



Diffuse Subsurface 
Scattering 



Figure 3: A block diagram of our data processing pipeline. Blocks 
in grey are the parameters of our skin reflectance model 

face scanner. Then we measure the total skin reflectance L{x 0 ,G> 0 ) 
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in a calibrated face-scanning dome by taking photographs from dif- 
ferent viewpoints under varying illumination. We refine the geom- 
etry and compute a high-resolution face mesh and a normal map 
using photometric stereo methods. To map between 3D face space 
and 2D (m, v) space we use the conformal parameterization of Shef- 
fer et al. [2005]. The data is densely interpolated using push-pull 
interpolation into texture maps of 2048 x 2048 resolution. 

The reflectance measurements are registered to the 3D geometry 
and combined into lumitexels [Lensch et al. 2001]. Each lumitexel 
stores samples of the skin BRDF /r(* 0 ,G) ( ,G) 0 ). We estimate the 
diffuse reflectance a(x 0 ) of f r using a stable minimum of lumitexel 
samples. Note that this diffuse reflectance encompasses the effects 
of the diffuse subsurface scattering Rd{r) and the modulation tex- 
ture M(x 0 ). We store a(x 0 ) in a diffuse albedo map and subtract 
±F t (ri,(Di)a(x 0 )F t (ri,(D 0 ) from f r {x 0 ,(Oi,(D 0 ). To the remaining 
specular reflectance f s (x 0 , we (it an analytic BRDF. 

The diffuse subsurface reflectance Rd(r) and the modulation tex- 
ture M (x 0 ) cannot be measured directly on human subjects. Instead, 
we measure their combined effect R(r) at a few locations in the face 
using a special contact device. To our measurements R(r) we fit the 
analytic dipole approximation of Jensen et al. [2001] and store the 
reduced scattering coefficient o* s and the absorption coefficient <r fl . 
Similar to Donner and Jensen [2005] we derive that: 



R d (r)M(x 0 ) = R(r) 



CC(Xg) 

CCavg 



(7) 



where ctavg is the average albedo. We use the albedo map a(x 0 ), 
the average of all albedo map values <Xa Vg , and the dipole approxi- 
mation for R(r) to compute R ( j(r)M(x 0 ) in Equation (6). 

In summary, the parameters for each face are the spatially- 
varying coefficients of the specular BRDF f s , an albedo map with 
diffuse reflectance values oc(jc 0 ), and the values for qj and o a . Sec- 
tion 6 shows how we use this data for rendering. We now describe 
each of these measurement and processing steps in more detail. 



5 Measurement System 

Figure 4 shows a photograph of our face-scanning dome. The sub- 




Figure 4: The face-scanning dome consists of 16 digital cameras, 
150 LED light sources, and a commercial 3D face-scanning system. 

ject sits in a chair with a headrest to keep the head still during the 
capture process. The chair is surrounded by 16 cameras and 150 
LED light sources that are mounted on a geodesic dome. The sys- 
tem sequentially turns on each light while simultaneously capturing 
images with all 16 cameras. We capture high-dynamic range (HDR) 




Figure 5: Facial detail shown in closeups. Left: Measured geom- 
etry before correction. Middle: Corrected high-quality geometry. 
No normal map was used; the surface detail is due to the actual 
geometry of the model. Right: Comparison photo. 

images [Debevec and Malik 1997] by immediately repeating the 
capture sequence with two different exposure settings. The com- 
plete sequence takes about 25 seconds for the two passes through 
all 150 light sources (limited by the frame rate of the cameras). To 
minimize the risk of light-induced seizures we ask all subjects to 
close their eyes. We report more details about the system and its 
calibration procedure in [Weyrich et al. 2005]. 

A commercial face-scanning system from 3QTech 
(www.3dmd.com) is placed behind openings of the dome. 
Using two structured light projectors and four cameras, it captures 
the complete 3D face geometry in less than one second. The output 
mesh contains about 40,000 triangles and resolves features as 
small as 1 mm. We clean the output mesh by manually cropping 
non-facial areas and fixing non-manifold issues and degenerate tri- 
angles. The cleaned mesh is refined using Loop-subdivision [Loop 
1987J to obtain a high-resolution mesh with about 700,000 to 
900,000 vertices. The subdivision implicitly removes noise. 

Next, we compute a lumitexel [Lensch et al. 2001 ] at each vertex 
from the image reflectance samples. The lumitexel radiance val- 
ues L(x o ,(0 o ) are normalized by the irradiance E(x o ,(0j) to obtain 
BRDF sample values: 



fr{x 0 ,(Di,(0 o ) = 



L{x 0 ,(o 0 ) = _____ 



(8) 



We calibrate the radiance of each light source using a white 
ideal diffuse reflector (Fluorilon) and explicitly model the beam 
spread by a bi-variate 2 nd degree polynomial [Weyrich et al. 2005]. 
We use shadow mapping to determine the visibility of lumitexel 
samples for each light source and camera. On average, a lumitexel 
contains about 900 BRDF samples per color channel, with many 
lumitexels having up to 1,200 samples. The numbers vary depend- 
ing on the lumitexel visibility. All processing is performed on RGB 
data except where noted otherwise. 

5.1 Geometry Processing 

We estimate normals at each lumitexel from the reflectance data 
using photometric stereo [Barsky and Petrou 2001]. We compute 
a normal map for each of the camera views. Using the method by 
Nehab et al. [2005] we correct. low-frequency bias of the normals. 
For each camera, we also compute the cosine between it and each 
normal and store the result as a blending weight in a texture. To 
compute the final diffuse normal map y we average all normals at 
each vertex weighted by their blending weights. To avoid seams 
at visibility boundaries between cameras, we slightly smooth the 
blending texture and the diffuse normal map using a Gaussian filter. 
Finally, we compute the high-resolution geometry by applying the 
method of Nehab et al. Figure 5 compares the measured 3QTech 
geometry to the final high-resolution mesh. 

Figure 6 (a) shows a rendering of a face using the Torrance- 
Sparrow BRDF model and the diffuse normal map. The rendering 
does not accurately reproduce the specular highlights that are vis- 
ible in the input photo in Figure 6 (d). For example, compare the 
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Figure 6: Torrance-Sparrow model and (from left to right) (a) only 
diffuse normal map; (b) one diffuse and one global micro-normal 
map for specular reflection; (c) one diffuse and per-view micro- 
normal maps; (d) input photograph. Method (c) most accurately 
reproduces the specular highlights of the input image. 

detailed highlights on the cheeks in (d) with the blurred highlight 
in (a). These highlights are due to micro-geometry, such as pores 
or very fine wrinkles, which is not accurately captured in the face 
geometry or diffuse normal map. 

To improve the result, we use micro-normals to render the spec- 
ular component. Debevec et al. [Debevec et al. 2000] used micro- 
normals for a Torrance-Sparrow model on their specular term. They 
estimated improved normals using a specular reflection image ob- 
tained using cross-polarized lighting. For each camera viewpoint, 
we choose the sample with maximum value in each lumitexel. The 
half-way vector between the view direction and the sample's light 
source direction is the micro-normal for this view. We either store 
these normals in per-camera micro-normal maps, or generate a sin- 
gle micro-normal map using the blending weights from before. Fig- 
ures 6 (b) and (c) show renderings with single or per-view micro- 
normal maps, respectively. Figure 6 (b) is better than (a), but only 
Figure 6 (c) accurately reproduces the highlights of the input im- 
age. The results in this paper were computed using per-view micro- 
normals. For between-camera views and animations we interpolate 
the micro-normals with angular interpolation [Debevec et al. 1996]. 

5.2 Diffuse Albedo Estimation 

We estimate the total diffuse reflectance a at each surface point 
from the lumitexel data. We assume that surface reflectance van- 
ishes for at least some of the observation angles, revealing pure 
diffuse reflectance. Accordingly, the total diffuse reflectance is 
the maximum a that is compliant with the observations, i.e., 
([/n)a does not exceed the Fresnel-normalized BRDF samples 
/ r /(F / (?7,Q)/)/ r f (r|,CD 0 )). We compute a as the minimum ratio be- 
tween this Fresnel-normalized f r and the unit diffuse reflectance: 



a(x 0 ) = min 



nf r {x 0 ,<D 0 ,<di) 
Ft(r),<0i)F t {j},(0 o ) ' 



(9) 



In order to reduce outliers and the influence of motion artifacts, 
we determine a stable minimum by penalizing grazing observations 
and discarding the k smallest values. The a values for each surface 
point are re-parameterized, interpolated, and stored as the diffuse 
albedo map. 

5.3 BRDF Fit 

The specular reflection is computed at each lumitexel by subtracting 
the total diffuse reflectance: 

/,(**, (0 o , (Oi) = f r (x 0 , <o 0 , ©/) - H^If, (tj , <Oi)F t (7\ , cub). (10) 

Highlights on dielectric materials like skin are of the same color 
as the light source (white, in our case). Consequently, we convert 
the specular reflectance to grey scale to increase the stability of the 
BRDF fit. Unlike previous work [Marschner et al. 1999; Lensch 



et al. 2001 ; Fuchs et al. 2005b] we do not need to cluster reflectance 
data because our acquisition system collects enough samples for a 
spatially-varying BRDF fit at each vertex. The data for lumitexels 
with a badly conditioned fit is interpolated during creation of the 
BRDF coefficient texture. 

We fit the Blinn-Phong [Blinn 1977] and Torrance-Sparrow [Tor- 
rance and Sparrow 1967] isotropic BRDF models to the data. The 
Blinn-Phong model is the most widely used analytic reflectance 
model due to its simplicity: 



fsBP=Ps^~ COS" 8. 



(ID 



Here 8 is the vector between the normal N and the half-way vector 
H, p s is the scaling coefficient, and n is the specular exponent. The 
factor (ai + 2)/27T is for energy normalization so that the cosine lobe 
always integrates to one. This assures that 100% of the incident 
energy is reflected for p s = 1 . 

We also use the physically-based Torrance-Sparrow model: 



1 

fsTS = Ps~ 



DG 



K {N • <0i){N ■ C0 0 ) 



with: 



2(N H)(N-C0 o ) 2(AMf)(JV-ffl t ) 1 
C = mm{1 ' (OV//) ' (ovtf) >' 



(12) 



(13) 



D = 



1 



m 2 cos 4 5 



exp -((tan«)/m]' 



C is the geometry term, D is the Beckmann micro-facet distribution, 
and F r is the reflective Fresnel term. The free variables are the 
scaling coefficient p s and the roughness parameter m. 

For the fit we use the fitting procedure by Ngan et al. [2005]. The 
objective function of the optimization is the mean squared error 
between the measured BRDF f s and the target model f S M (either 
fsBP or f S Ts) w ^h parameter vector p: 



£(P) 



r s((Qh(Oo)cos Si - f sM {<XH, (0 0 \V) cos ( 



(14) 



The sum is over the non-empty samples of the lumitexel, and 0, is 
the elevation angle of incident direction. The weight w is a correc- 
tion term that allows us to ignore data with o>; or (0 o greater than 
80°. Measurements close to extreme grazing angles are generally 
unreliable due to imprecisions in the registration between 3D geom- 
etry and camera images. We apply constrained nonlinear optimiza- 
tion based on sequential quadratic programming over the specular 
lobe parameters to minimize the error metric. 

To assess the quality of our fit, we compare renderings from cam- 
era viewpoints to the actual photographs. The camera and light 
source calibration data were used to produce identical conditions 
in the renderings. Figure 7 shows a comparison of the original 
and synthesized images with the same illumination and different 
BRDF models. The spatially-varying Blinn-Phong model (c) over- 
estimates the specular component and does not match the photo- 
graph as closely as the spatially- varying Torrance-Sparrow model 
(b). Nevertheless, it produces more accurate results compared to 
the spatially uniform Torrance-Sparrow BRDF (d). 

Another popular choice for BRDF fitting is the Lafortune 
model [Lafortune et al. 1997] because it models off-specular re- 
flections while still being computationally simple. Because the 
non-linear Lafortune model has more parameters than the Torrance- 
Sparrow and Blinn-Phong models, fitting of the model becomes 
more demanding, even for a single-lobe approximation. A realistic 
employment of the Lafortune model would use at least three lobes. 
This fit has been reported by [Ngan et al. 2005] to be very unstable 
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(a) (b) (c) (d) 

Figure 7: (a) Input photograph compared to our face model with (b) spatially-varying Torrance-Sparrow; (c) spatially-varying Blinn-Phong; 
and (d) uniform Torrance-Sparrow BRDF models. 



compared to other reflectance models. In addition, [Lawrence et al. 
2004] explain that "Often, a nonlinear optimizer has difficulty fit- 
ting more than two lobes of a Lafortune model without careful user 
intervention." Given the high quality that we could achieve with the 
Torrance-Sparrow model, we decided not to further investigate the 
Lafortune model. 

5.4 Measuring Translucency 

Our subsurface measurement device is an image-based version of 
a fiber optic spectrometer with a linear array of optical fiber de- 
tectors [Nickell et al. 2000] (see Figure 8). Similarly, more so- 



Sensor Head Detector Housing 




Figure 8: Left: A picture of the sensor head with linear fiber array. 
The source fiber is lit. Right: The fiber array leads to a camera in 
a light-proof box. The box is cooled to minimize imaging sensor 
noise. 

phisticated devices called Laser Doppler Flowmetry (LDF) probes 
are used to measure micro-circulatory blood flow and diffuse re- 
flectance in skin [Larsson et al. 2003]. The advantage of our device 
over LDF or the measurement procedure in [Jensen et al. 2001] 
is that it can be safely used in faces. Light from a white LED is 
coupled to a source fiber. The alignment of the fibers is linear to 
minimize sensor size. A sensor head holds the source fiber and 
28 detection fibers. A digital camera records the light collected by 
the detector fibers. The camera and detector fibers are encased in 
a light-proof box with air cooling to minimize imager noise. We 
capture 23 images bracketed by 2/3 f-stops to compute an HDR im- 
age of the detector fibers. The total acquisition time is about 88 
seconds. 

Figure 9 shows the sensor head placed on a face. We have chosen 
to measure three points where the sensor head can be placed reli- 
ably: forehead, cheek, and below the chin. For hygienic reasons we 
do not measure lips. We found that pressure variations on the skin 
caused by the mechanical movement of the sensor head influence 
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Figure 9: Left: The sensor head placed on a face. Top: Sensor fiber 
layout. The source fiber is denoted by a cross. Bottom: An HDR 
image of the detector fibers displayed with three different exposure 
values. 

the results. To maintain constant pressure between skin and sen- 
sor head we attached a silicone membrane connected to a suction 
pump. This greatly improves the repeatability of the measurements. 
For more details on the subsurface device and calibration procedure 
see [Weyrich et al. 2005]. 

Previous work in diffuse reflectometry [Nickell et al. 2000] sug- 
gests that some areas of the human body exhibit anisotropic subsur- 
face scattering (e.g., the abdomen). We measured two-dimensional 
subsurface scattering on the abdomen, cheek, and forehead for a 
few subjects. We verified the presence of significant anisotropy in 
the abdominal region (see Figure 10). However, the plots show 



Figure 10: Subsurface scattering curves for abdomen, cheek, and 
forehead measured along 16 ID profiles. 



that the diffuse subsurface scattering of facial skin can be well ap- 
proximated with an isotropic scattering model. Consequently, we 
measure only a one-dimensional profile and assume rotational sym- 
metry. 

We fit the analytic BSSRDF model of Jensen et al. [2001] to 
the data points of each subsurface measurement, providing us with 
the reduced scattering coefficient a' s and absorption coefficient o a . 
From the measured a a and & s data we can derive the effective trans- 
port coefficient: 

<*r=^/3a«(<fc + oO*l/V (15) 
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id is the diffuse mean free path of photons (mm) and provides a 
measure of skin translucency. 

Table 1 shows the mean and variance of a tr for a sample popula- 
tion of various people (see Section 7). The measurement points on 



BRDF approximation by Jensen et al. [2001]: 



(mm~ l ) 


Cheek 
Mean Van 


Forehead 
Mean Van 


Neck 
Mean Van 


red 

green 

blue 


0.5572 0.1727 
0.9751 0.2089 
1.5494 0.1881 


0.5443 0.0756 
0.9831 0.1696 
1.5499 0.2607 


0.6911 0.2351 
1.2488 0.3686 
1.9159 0.4230 



Table 1: Mean and variance of a tr in our dataset. 

cheek and forehead are quite similar in translucency. The measure- 
ment point on the neck underneath the chin shows a rather different 
mean, but also higher variance. This is probably due to measure- 
ment noise, as the sensor head is hard to place there. 

Figure 1 1 shows closeups of the subjects with minimum 
[1.0904,0.6376,0.6035] and maximum [2.8106,1.2607,0.6590] 
translucency values t& in our dataset. There are subtle differences 




Figure 11: Photographs of subjects with minimum (left) and maxi- 
mum (right) translucency values in our dataset. The differences at 
shadow boundaries are subtle. 

visible at shadow boundaries. Figure 12 shows closeups computed 
with our model using the same minimum and maximum translu- 
cency values on one of our face models. Note that the model is 



) 
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Figure 12: Synthetic images with minimum (left) and maximum 
(right) translucency values. 

capable of reproducing the subtle differences of Figure 11. Translu- 
cency values 1^ do not vary much between measurement points 
and between individuals, which allows us to approximate o tr with 
an average value a trAvg = [0.5508,0.9791, 1.5497] for all subjects. 
This is the mean over all subjects of the measurements for the fore- 
head and cheek, ignoring the neck. 

As observed by [Larsson et al. 2003], it is difficult to reliably 
measure the spatially-varying absorption coefficient a a with a con- 
tact device. However, using a tr Av g we can estimate spatially uni- 
form scattering parameters O a and a' s from skin albedo without any 
direct measurements of subsurface scattering. We use the diffuse 



(16) 

This approximation is valid for planar, homogeneously scattering 
materials under uniform directional illumination. It is quite accu- 
rate for the average total diffuse reflectance Q^ v ^» which is the av- 
erage of all albedo map values a{x Q ). Using Sbrdf = Qavg, we 
compute the apparent albedo a' by inverting Equation (16) using 
a secant root finder. Wc derive the model parameters o' s and a a 
from a' and o t rAvg using [Jensen and Buhler 2002] o' s = a!a[ and 

o a = o' t - a f sy with al = o t rAvgl \J?>{1 - a'). These diffuse scat- 
tering parameters are not spatially varying because we use Oa Vg to 
compute them. Instead, we capture the spatial variation of diffuse 
reflectance using the modulation texture M(x Q ). 

6 Rendering 

We implemented our reflectance model using a high-quality Monte 
Carlo offline ray tracer. The direct surface reflectance was eval- 
uated using the Torrance-Sparrow BRDF model. The subsurface 
reflectance was computed using the diffusion dipole method as de- 
scribed in [Jensen et al. 2001] and modulated by the modulation tex- 
ture M (x 0 ). M(x a ) can be derived from Equation (7) as described in 
Section 4. Each image took approximately five minutes to render on 
a single Intel Xeon 3.06GHz workstation. Figure 13 shows compar- 
isons between real and synthetic images for different faces and dif- 
ferent viewpoints. The synthetic and real images look very similar, 
but not absolutely identical. The main limitation is a lack of sharp- 
ness, both in the texture and in geometry, mainly because accuracy 
in geometric calibration and alignment remain issues. The inten- 
sity and shape of the specular highlights in the synthetic images is 
sometimes underestimated. The shape of the specular highlights 
- especially on the forehead - is greatly affected by fine wrinkles 
and small pores. Several sources of error (measured 3D geometry, 
motion artifacts, calibration errors, camera noise, Bayer interpola- 
tion, errors in the photometric stereo estimations, etc.) prevent us 
from capturing the micro-normal geometry with 100% accuracy. 
Much higher-resolution and higher-quality face scans can be ob- 
tained by creating a plaster mold of the face and scanning it with a 
high-precision laser system. For example, the XYZRGB 3D scan- 
ning process originally developed by the Canadian National Re- 
search Center contributed significantly to the realism of the Disney 
project [Williams 2005] and the Matrix sequels. However, it would 
be prohibitive to scan more than one hundred subjects this way. It 
would also be difficult to correspond the 3D geometry with the im- 
age data from the face scanning dome due to registration errors, 
subtle changes in expressions, or motion artifacts. In addition, the 
molding compound may lead to sagging of facial features [Williams 
2005] . We believe that our system strikes a good tradeoff between 
speed, convenience, and high image quality. 

Facial hair is currently not explicitly represented in our model. 
Although the overall reflectance and geometry of the bearded per- 
son in Figure 13 has been captured, the beard is lacking some details 
that are visible in the photograph. We avoided scanning people with 
full beards or mustaches. 

We model subsurface scattering using a single layer. However, 
skin consists of several layers that have different scattering and ab- 
sorption properties [Tuchin 2000]. The multi-layer dipole model 
of Donner et al. [2005] would provide a better approximation, but 
it is not clear how to measure its parameters in vivo for people of 
different skin types. 

The dipole diffusion approximation overestimates the total flu- 
ence in the superficial areas of skin, making the skin appear slightly 
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CL-irt 

oKin 


dKin color 


Sun Exposure Reaction 


Subjects 


TVpe 






(M/F) 


I 


Very white 


Always burn 


N/A 


II 


White 


Usually burn 


8/6 


III 


White to olive 


Sometimes burn 


49/18 


IV 


Brown 


Rarely burn 


40/8 


V 


Dark brown 


Very rarely burn 


13/2 


VI 


Black 


Never burn 


4/1 



Table 2: The Fitzpatrick skin type system and the number of sub- 
jects per skin type. 
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Figure 14: Plot of a a [mm] and a' s [mm] for all subjects. Color 
encodes skin type (light = //, black = VI); shape encodes gender 
(& = F f o = M); marker size encodes age (larger = older). Skin 
absorption is highest for skin with low scattering and decreases for 
more highly scattering skin. 

more opaque than actual skin [Igarashi et al. 2005]. In addition, our 
measurements are confined to RGB data. For example, the para- 
meters <T fl and a f s for skin are wavelength dependent [Igarashi et al. 
2005]. Krishnaswamy and Baranoski [2004] developed a multi- 
spectral skin model and emphasize the importance of spectral ren- 
dering in the context of skin. 

7 Face Reflectance Analysis 

We measured face reflectance for 149 subjects (1 14 male / 35 fe- 
male) of different age and race. The distribution by age is (M/F): 
20s or younger (61/18), 30s (37/10), 40s (10/2), 50s or older (6/5). 
Because race is an ambiguous term (e.g., some Caucasians are 
darker than others), we classify the data by skin type according to 
the Fitzpatrick system [Fitzpatrick 1988]. Since we measured only 
two individuals of skin type I, we group types I and II together. 
Table 2 lists the number of subjects per skin type. 

Analysis of Absorption and Scattering Parameters: We al- 
ready discussed the measured subsurface scattering data in Sec- 
tion 5.4. The absorption coefficient a a and the reduced scattering 
coefficient o' s represent the amounts of light attenuation caused by 
absorption and multiple scattering, respectively. We estimate these 
parameters from the average albedo a avg and the average translu- 
cency o t rAvg- Figure 14 shows the distribution of absorption and 
reduced scattering values for all subjects. Skin type is encoded by 
color (light = II, black = VI), gender by marker type (o = F, o = M), 
and age by marker size (larger = older). Subjects with skin type 
V and VI have higher absorption and lower scattering coefficients 
than subjects with skin type II or III. There seems to be no correla- 
tion with age. As expected, the absorption coefficient a a is highest 
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Figure 15: Left: 10 face regions. Right: Variation of Torrance- 
Sparrow parameters per face region averaged over all subjects. The 
center of the ellipse indicates the mean. The axis of each ellipse 
shows the directions of most variation based on a PCA. 

for the blue and green channels. In addition, the range of a' s is much 
smaller for the green channel. These effects are due to the absorp- 
tion of blue and green light by blood in the deeper levels of the skin. 
The average absorption o a decreases in people with higher scatter- 
ing coefficient a[. The relationship is near-exponential since we 
are using the diffuse BRDF approximation by Jensen et al. (Equa- 
tion (16)). 

Analysis of Torrance- Sparrow BRDF Parameters: To 

study the variations of the specular reflectance we analyzed the 
physically-based Torrance-Sparrow model parameters. Similar to 
the work by Fuchs et al. [Fuchs et al. 2005b], we distinguish the 
10 different face regions shown in Figure 15 (left). For each re- 
gion, we randomly chose a few thousand vertices by marking them 
manually. We collected the BRDF parameters for the vertices and 
averaged them per region. Figure 15 (right) visualizes the resulting 
parameter distributions, averaged over all subjects. To create this 
visualization, we computed a 2 x n matrix for each region, where n 
is the total number of samples in that region for all subjects. We 
then performed a Principal Component Analysis (PCA) on each 
matrix. The resulting eigenvectors and eigenvalues are the direc- 
tions and size of the ellipse axis, respectively. The centers of the 
ellipses have been translated to the mean values of the BRDF para- 
meters for that region. 

As shown in Figure 15, the vermilion zone (or red zone) of the 
lips (8) exhibits a large parameter variation. Lips have smooth, 
glabrous skin, i.e., they have no follicles or small hairs. The epi- 
dermis is thin, almost transparent, and the blood vessels of the der- 
mis are close to the surface, giving raise to the red appearance of 
the lips. The reflectance variations are most likely due to circula- 
tion differences in these blood vessels and the highly folded der- 
mis, which results in the ridged appearance of the lips. Big pa- 
rameter variations can also be observed in regions 1 (mustache), 
3 (eyebrows), 9 (chin), and 10 (beard). Male subjects contain fa- 
cial hair in these regions, which leads to noisy normal estimations 
and widely varying parameter distributions. Smoother facial areas, 
such as the forehead (2), eyelids (4), nose (6), and cheeks (7), show 
much smaller parameter variance. The highest specular scaling co- 
efficients p s appear for nose (6), eyebrows (3), eyelids (4), lips (8), 
and forehead (2). The nose and forehead especially are areas of 
high sebum secretion, making them appear more specular. Not sur- 
prisingly, the nose (6) shows the smallest values for the roughness 
parameter m because it lacks small facial wrinkles. 

To analyze how the Torrance-Sparrow parameters vary with skin 
type, age, and gender, we computed the average of the BRDF pa- 
rameters in each region and stored the data for all subjects in a 
23 x 149 matrix M. The rows of the matrix contain the average 
BRDF parameters for each face region, the gender [0 = M , 1 = F] 9 



1021 




Figure 16: Canonical correlation analysis of the Torrance- 
Sparrow BRDF parameters. Color encodes skin type ( light = II, 
black - VI); shape encodes gender (o = F, o = M); marker size en- 
codes age (larger = older). Skin type is the only reliable predictor 
of BRDF parameters. 

the age, and the skin type [// - VI]. The columns contain the data 
for each person. To visualize the matrix M we tried linear (PCA) 
and non-linear dimensionality reduction techniques. However, the 
data is tightly clustered in a high-dimensional ball that cannot be 
reduced to fewer dimensions. Instead, we use Canonical Correla- 
tion Analysis (CCA) [Green et al. 1966]. CCA finds projections 
of two multivariate datasets that are maximally correlated. Fig- 
ure 16 shows the output of CCA for the Torrance-Sparrow model. 
Here the data is projected (orthogonally) onto the two dimensions 
that are most strongly correlated with the experimental variables of 
skin type, gender, and age. The first dimension is best predicted 
by skin type (color), the second by gender (o = F, o = M). Age is 
encoded in marker size. The principal correlations are 0.768 and 
0.585 (to mixtures of the experimental variables); the direct corre- 
lations are 0.766 for skin type, 0.573 for gender, and 0.537 for age. 
This indicates that CCA was not able to greatly improve the direct 
correlations. Maybe not surprisingly, skin type is the only reliable 
predictor for BRDF parameters. 

To analyze variations of BRDF parameters due to external fac- 
tors, we also measured one male subject under several external 
conditions (normal, cold, hot, sweaty, with makeup, with lotion, 
and with powder). As expected, there are noticeable differences 
between these BRDFs, especially between lotion / hot and cold / 
powder. 

Dissemination: The statistics of our face reflectance analysis of 
all subjects will be made public to the research community. We are 
releasing all reflectance data, such as BRDF coefficients, absorption 
and scattering coefficients, and albedo histograms, for each subject. 
In addition, we are providing an interactive applet that outputs av- 
erage parameter values and histograms based on user choice for 
gender, age, and skin type. This data will allow image synthesis, 
editing, and analysis of faces without having to acquire and process 
a large dataset of subjects. 

8 Face Editing 

Our face reflectance model has intuitive parameters for editing, 
such as BRDF parameters, average skin color, or detailed skin tex- 
ture. To change the surface reflectance of a model, we can either 



edit the parameters of the analytic BRDF model directly, or we can 
adjust them using the statistics from our database. For each face 
region, we can match the parameter histogram of the source face 
with another parameter histogram using the histogram interpola- 
tion technique presented in [Matusik et al. 2005]. This allows us to 
change a face BRDF based on, for example, different external con- 
ditions (e.g., from normal to lotion) (see Figure 17, bottom). We 
can also transfer skin color histograms from one subject to another. 
This allows us to change, for example, the skin type (e.g., from II 
to V) (see Figure 1, right). 

To transfer skin texture, we use the texture analysis technique of 
Heeger and Bergen [1995]. We first transform the albedo map of 
each subject into a common and decorrelated color space using the 
method discussed in [Heeger and Bergen 1995][Section 3.5] and 
process each transformed color channel independently. We then 
compute statistics (histograms) of the original albedo map at full 
resolution, and of filter responses at different orientations and scales 
organized as a steerable pyramid. We use seven pyramid levels with 
four oriented filters, and down-sample the albedo map by a factor 
of two at each level. Each histogram has 256 bins. The first level 
of the pyramid is a low-pass and high-pass filter computed on the 
full-resolution albedo map. The output of the low-pass filter is a 
histogram of average skin color. 

The histograms of all pyramid filter responses (30 total) and 
the histogram of the original albedo map are concatenated into a 
256 x 31 x 3 = 23,808 element vector H. To transfer skin tex- 
tures, we use the histogram vector H of a source and a target 
face and apply the histogram-matching technique of Heeger and 
Bergen [1995]. We can either use the histograms of the whole 
albedo map or restrict the analysis/synthesis to certain facial re- 
gions. Note that Heeger and Bergen start their texture synthesis 
with a noise image as the source. In our case it makes more sense 
to start from the source albedo map. To allow for sufficient variation 
during histogram matching, we add some noise to the source albedo 
map before we compute its histogram vector H. A similar method 
was used by Tsumura et al. [2003] for synthesis of 2D skin textures, 
and by Cula and Dana [2002] to analyze bi-directional texture func- 
tions (BTFs) of skin patches. Figure 17 shows transfers of skin tex- 
ture, BRDF, and skin color for two face models. The translucency 
values of the source and result models remain the same. The exam- 
ple in Figure 1 (right) shows skin albedo and BRDF transfers. The 
resulting face appears lighter and shinier. 



9 Conclusions and Future Work 

In this paper we have proposed a simple and practical skin model. 
An important feature of our model is that all its parameters can 
be robustly estimated from measurements. This reduces the large 
amount of measured data to a manageable size, facilitates editing, 
and enables face appearance changes. Images from our model come 
close to reproducing photographs of real faces for arbitrary illumi- 
nation and pose. We fit our model to data of a large and diverse 
group of people. The analysis of this data provides insight into the 
variance of face reflectance parameters based on age, gender, or 
skin type. We are making the database with all statistics available 
to the research community for face synthesis and analysis. 

As discussed in Section 6, there is room for improvement in 
our model. For example, it would be interesting to measure 
wavelength-dependent absorption and scattering parameters. It 
would also be interesting to compare the results from the diffuse 
dipole approximation with a full Monte Carlo subsurface scattering 
simulation. Other important areas that require a different model- 
ing approach are facial hair (eyebrows, eyelashes, mustaches, and 
beards), hair, ears, eyes, and teeth. Very fine facial hair also leads 
to asperity scattering and the important "velvet" look of skin near 
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Figure 17: Face appearance changes. Top: Transferring the freck- 
led skin texture of the subject in Figure 1 (left). Bottom: Changing 
the skin type (BRDF and albedo) from type III to type IV. 

grazing angles [Koenderink and Pont 2003]. Our model does not 
take this into account. 

We captured face reflectance on static, neutral faces. Equally 
important are expressions and face performances. For example, 
it is well known that the blood flow in skin changes based on fa- 
cial expressions. Our setup has the advantage that such reflectance 
changes could be captured in real-time. 
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